Certification of Autonomous Systems under UK Military Safety Standards

There is growing interest in many sectors in developing highly autonomous systems such as Unmanned Air Vehicles (UAVs) with significant in-mission executive power. Such systems have the potential to replace humans in a variety of dangerous tasks, but there is concern that the combination of novel technologies with demanding tasks and unpredictable environments will lead to new safety challenges. This paper reviews proposed scenarios of autonomous system applications and identifies the safety concerns that they raise. It then explores how autonomous systems can be certified as safe to operate within the terms of the existing safety standards that are used by the UK military. The combination of difficulties arising from the nature of autonomous systems and the provisions of the existing safety standards raise serious concerns about the practicality of certification. With this in mind, promising work on new techniques for analysing and ensuring the safety of autonomous systems is reviewed. Finally, some future concerns and directions are identified.